Intelligent traffic quota management

ABSTRACT

A network element acts as a gateway to a data network for a subscriber end station. The network element includes control plane operable to communicate with a first network processing unit (NPU) and a second NPU, which are operable to communicate with the subscriber end station. The control plane includes a quota management module, which determines a quota amount to be assigned to the first NPU and the second NPU. The quota management module assigns a portion of the quota amount to the first NPU and another portion of the quota amount to the second NPU. The quota management module may determine to change the distribution of an unconsumed quota amount between the first NPU and the second NPU, determine the unconsumed quota amount, and assign a portion of the unconsumed quota amount to the first NPU and another portion of the unconsumed quota amount to the second NPU.

FIELD

Embodiments of the invention relate to traffic quota management inrouting and gateway platforms. Specifically, embodiments of theinvention relate to a method and device for distributing available quotaamounts appropriately and efficiently between ingress and egress networkprocessing units.

BACKGROUND

Subscribers of voice and/or data services provided by service providersutilize subscriber end stations such as mobile electronic devices toconnect to the service provider's wireless or wireline network. Thecommunication between the subscriber end station and the serviceprovider's network occurs through one or more base stations using one ormore standards or protocols, such as those defined by the 3rd GenerationPartnership Project (3GPP). One current 3GPP standard for mobilesubscriber end stations is Long Term Evolution (LTE), which is awireless standard for high-speed data communication. The voice and/ordata services provided by such networks are typically tracked usingonline charging functions to monitor the amount of various types oftraffic used by each subscriber end station. Such charging functionshave been presented in standards put forth by the 3GPP.

In these communication networks, a gateway, which provides thesubscriber with a connection to a particular network resource, mayutilize subscriber information (e.g., policy configuration information,traffic policing information, etc.) provided by an external server suchas an Online Charging Server. This charging information may includeinformation used for monitoring the subscriber's traffic consumption forpolicy enforcement purposes (e.g., throttle peer-to-peer traffic,throttle streaming media traffic, etc.) and/or for accounting andcharging the subscriber (e.g., charging based upon the content oftraffic, enforcing a quota for a type of content based upon creditassigned to the subscriber, etc.).

When a gateway is to enforce a quota for a particular traffic flow, itreceives a quota amount from the external server. This quota amountrepresents a total amount of data that may be sent to and received froma subscriber. However, some gateways may utilize a forwarding planecontaining dedicated network processing units. For example, a gatewaymay contain one network processing unit that only processes uplinktraffic and one network processing unit that only processes downlinktraffic. In such gateways, it is difficult to appropriately split thereceived quota amount into two portions and minimize the need to signalthe external server for additional quota amounts.

SUMMARY

According to one embodiment of the invention, a network element coupledbetween a data network and a subscriber end station acts as a gateway toa data network for the subscriber end station and performs intelligentquota management of one or more subscriber flow quotas that monitor theamount of traffic sent between the data network and the subscriber endstation. The network element includes a first network processing unit(NPU) and a second NPU that are operable to communicate with thesubscriber end station. The network element also includes a controlplane that is operable to communicate with the first NPU and the secondNPU. The control plane includes a quota management module, whichdetermines a quota amount associated with one of one or more trafficflows to be assigned to the first NPU and the second NPU. The quotamanagement module also assigns a portion of the quota amount to thefirst NPU and another portion of the quota amount to the second NPU. Thequota management module is further operable to determine to change thedistribution of an unconsumed quota amount between the first NPU and thesecond NPU. The quota management module determines the unconsumed quotaamount, and assigns a portion of the unconsumed quota amount to thefirst NPU and another portion of the unconsumed quota amount to thesecond NPU. This network element is capable to perform efficient quotadistribution and redistribution between the first NPU and second NPUproviding reduced internal control messaging and reduced signaling to anexternal policing and charging control node.

According to another embodiment of the invention, a method in a networkelement acting as a gateway in a communications network for intelligentquota management of a set of one or more subscriber flow quotasassociated with a subscriber end station, each of the set of quotas forpolicing the amount of traffic travelling between the network elementand the subscriber end station in one or more traffic flows, includesthe step of assigning a first quota portion to a first NPU. The firstquota portion is a portion of one of the set of quotas associated withone of the one or more traffic flows. The method further includes thestep of assigning a second quota portion to a set of one or more otherNPUs. The second quota portion is another portion of the one of the setof quotas. The method further includes the steps of determining tochange the distribution of an unconsumed quota amount of the one of theset of quotas, and determining the unconsumed quota amount of the one ofthe set of quotas. The method further includes the step of assigning athird quota portion to the first NPU. The third quota portion is aportion of the unconsumed quota amount of the one of the set of quotas.The method further includes the step of assigning a fourth quota portionto the set of other NPUs. The fourth quota portion is another portion ofthe unconsumed quota amount of the one of the set of quotas. This methodprovides for efficient quota distribution and redistribution between aplurality of NPUs, reduced internal control messaging, and reducedsignaling to an external policing and charging control node.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may best be understood by referring to the followingdescription and accompanying drawings that are used to illustrateembodiments of the invention. In the drawings:

FIG. 1 illustrates an exemplary system including a network elementutilizing multiple network processing units according to an embodimentof the invention;

FIG. 2 illustrates a flow diagram of a method in a network element forassigning and redistributing quota between a set of network processingunits according to an embodiment of the invention;

FIG. 3 illustrates a flow diagram of a method in a network element fordetermining an unconsumed quota amount according to an embodiment of theinvention;

FIG. 4 illustrates a flow diagram of a method in a network element fordetermining an unconsumed quota amount according to an embodiment of theinvention;

FIG. 5 illustrates a flow diagram of a method in a network element forquota redistribution according to an embodiment of the invention;

FIG. 6 illustrates a flow diagram of a method in a network element forquota redistribution according to an embodiment of the invention;

FIG. 7 illustrates a sequence diagram depicting a quota managementprocess in a network according to an embodiment of the invention; and

FIG. 8 illustrates a sequence diagram depicting a quota managementprocess in a network according to an embodiment of the invention.

DESCRIPTION OF EMBODIMENTS

In the following description, numerous specific details are set forth.However, it is understood that embodiments of the invention may bepracticed without these specific details. In other instances, well-knowncircuits, structures and techniques have not been shown in detail inorder not to obscure the understanding of this description. Those ofordinary skill in the art, with the included descriptions, will be ableto implement appropriate functionality without undue experimentation.

References in the specification to “one embodiment,” “an embodiment,”“an example embodiment,” etc., indicate that the embodiment describedmay include a particular feature, structure, or characteristic, butevery embodiment may not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it is submitted that it is within the knowledge of oneskilled in the art to effect such feature, structure, or characteristicin connection with other embodiments whether or not explicitlydescribed.

In the following description and claims, the terms “coupled” and“connected,” along with their derivatives, may be used. It should beunderstood that these terms are not intended as synonyms for each other.“Coupled” is used to indicate that two or more elements, which may ormay not be in direct physical or electrical contact with each other,co-operate or interact with each other. “Connected” is used to indicatethe establishment of communication between two or more elements that arecoupled with each other.

Different embodiments of the invention may be implemented usingdifferent combinations of software, firmware, and/or hardware. Thus, thetechniques shown in the figures can be implemented using code and datastored and executed on one or more electronic devices (e.g., an endstation, a network element). Such electronic devices store andcommunicate (internally and/or with other electronic devices over anetwork) code and data using computer-readable media, such asnon-transitory computer-readable storage media (e.g., magnetic disks,optical disks, random access memory, read only memory, flash memorydevices, phase-change memory) and transitory computer-readabletransmission media (e.g., electrical, optical, acoustical or other formof propagated signals—such as carrier waves, infrared signals, digitalsignals). In addition, such electronic devices typically include a setof one or more processors coupled with one or more other components,such as one or more storage devices (non-transitory machine-readablestorage media), user input/output devices (e.g., a keyboard, atouchscreen, and/or a display), and network connections. The coupling ofthe set of processors and other components is typically through one ormore busses and bridges (also termed as bus controllers). Thus, thestorage device of a given electronic device typically stores code and/ordata for execution on the set of one or more processors of thatelectronic device.

As used herein, a network element (e.g., a router, switch, bridge) is apiece of networking equipment, including hardware and software, whichcommunicatively interconnects other equipment on the network (e.g.,other network elements, end stations). Some network elements are“multiple services network elements” that provide support for multiplenetworking functions (e.g., routing, bridging, switching, Layer 2aggregation, session border control, Quality of Service, and/orsubscriber management), and/or provide support for multiple applicationservices (e.g., data, voice, and video). Subscriber end stations (e.g.,servers, workstations, laptops, netbooks, palm tops, mobile phones,smartphones, multimedia phones, Voice Over Internet Protocol (VOIP)phones, user equipment, terminals, portable media players, GPS units,gaming systems, set-top boxes) access content/services provided over theInternet and/or content/services provided on virtual private networks(VPNs) overlaid on (e.g., tunneled through) the Internet. The contentand/or services are typically provided by one or more end stations(e.g., server end stations) belonging to a service or content provideror end stations participating in a peer to peer service, and mayinclude, for example, public webpages (e.g., free content, store fronts,search services), private webpages (e.g., username/password accessedwebpages providing email services), and/or corporate networks over VPNs.Typically, subscriber end stations are coupled (e.g., through customerpremise equipment coupled with an access network (wired or wirelessly))with edge network elements, which are coupled (e.g., through one or morecore network elements) with other edge network elements, which arecoupled with other end stations (e.g., server end stations).

Some network elements include functionality for AAA (authentication,authorization, and accounting) protocols (e.g., RADIUS (RemoteAuthentication Dial-In User Service), Diameter, and/or TACAS+ (TerminalAccess Controller Access Control System)). AAA can be provided through aclient/server model, where the AAA client is implemented on a networkelement and the AAA server can be implemented either locally on thenetwork element or on a remote end station (e.g., server end station)coupled with the network element. Authentication is the process ofidentifying and verifying a subscriber. For instance, a subscriber mightbe identified by a combination of a username and a password or through aunique key. Authorization determines what a subscriber can do afterbeing authenticated, such as gaining access to certain end stationinformation resources (e.g., through the use of access controlpolicies). Accounting is recording user activity. By way of a summaryexample, subscriber end stations may be coupled (e.g., through an accessnetwork) through an edge network element (supporting AAA processing)coupled with core network elements coupled with server end stations ofservice/content providers. AAA processing is performed to identify thesubscriber record for a subscriber. A subscriber record includes a setof attributes (e.g., subscriber name, password, authenticationinformation, access control information, rate-limiting information,policing information) used during processing of that subscriber'straffic.

In wireless communication networks, such as 2G/3G and 3rd GenerationLong Term Evolution/Evolved Packet Core (3G LTE/EPC) networks, similarsubscriber attribute information, such as rate-limiting information andpolicing information, may be managed by Policy Charging and RulesFunction (PCRF) nodes and/or Online Charging System (OCS) servers. Insuch networks, the network element may be a gateway deployed to providedata services to subscriber end stations that relies upon policyconfiguration or traffic policing information from an external server(e.g. OCS server, AAA server, PCRF node, etc.) to manage these dataservices. Examples of network elements serving as gateways includePacket Data Network Gateways (PDN-GW or PDN Gateway) in 3G LTE/EPCnetworks, Gateway GPRS Support Nodes (GGSN) in the packet switcheddomain of 2G/3G networks, and Broadband Remote Access Servers (BRAS) inwireline networks.

In a 3G LTE/EPC wireless communication network, a subscriber end stationmay connect to a PDN-GW to send or receive data with an externalpacket-switched network through an access node base station such as aneNodeB. From the eNodeB, via a S1-U interface, the connection may passthrough another network element known as a serving gateway (S-GW), whichroutes and forwards data. The S-GW, via an S5 interface, passes theconnection through to the PDN-GW, which acts as an IP point ofattachment for the subscriber end station. The PDN-GW may communicatewith an external server (e.g., OCS server, AAA server, PCRF node, etc.)for the purpose of obtaining policing and configuration information.This policing and configuration information may include information usedfor monitoring the subscriber's traffic consumption for policyenforcement purposes (e.g., to throttle peer-to-peer traffic, tothrottle streaming media traffic, etc.) and for accounting and chargingthe subscriber based upon the content of the traffic. For example, theconfiguration information may require that a traffic quota be enforcedon VOIP traffic of a mobile subscriber based upon credit availablewithin the subscriber's account. Similarly, the configurationinformation may require that a traffic quota be policed for streamingmedia traffic to limit its rate of transmission. Thus, quota or volumethresholds used for charging or policing traffic flows are downloadedfrom the external server to the PDN-GW to be used when providing dataservices to the subscriber end station. Typically, a quota threshold isa number that encompasses a cumulative threshold for both uplink anddownlink traffic that the PDN-GW may use while providing a particulartype of data service to the subscriber end station.

Network elements are commonly separated into a control plane and a dataplane (sometimes referred to as a forwarding plane or a media plane). Inthe case that the network element is a router (or is implementingrouting functionality), the control plane typically determines how data(e.g., packets) is to be routed (e.g., the next hop for the data and theoutgoing port for that data), and the data plane is in charge offorwarding that data. For example, the control plane typically includesone or more routing protocols (e.g., Border Gateway Protocol (BGP),Interior Gateway Protocol(s) (IGP) (e.g., Open Shortest Path First(OSPF), Routing Information Protocol (RIP), Intermediate System toIntermediate System (IS-IS)), Label Distribution Protocol (LDP),Resource Reservation Protocol (RSVP)) that communicate with othernetwork elements to exchange routes and select those routes based on oneor more routing metrics.

Some routing and gateway network elements utilize a forwarding planearchitecture that includes multiple network processing units. A networkelement forwarding plane may include a data card with one or more NPUs.Alternatively, a network element forwarding plane may include more thanone data card, each having one or more NPUs. Commonly, such systems useseparate directional processing units, wherein one or more dedicatedingress NPUs process uplink traffic from subscribers and one or morededicated egress NPUs process downlink traffic to the subscribers. Inother network elements, the multiple packet processing units need not bededicated to only ingress or egress traffic; each may process bothingress and egress traffic. However, when receiving traffic managementinformation from an external server (e.g., AAA server, OCS server, PCRFnode, etc.), the gateway is assigned a particular quota amount for aparticular flow of traffic for a subscriber. This quota amount is asingle amount of traffic allowed to be utilized by the subscriber forthe flow of traffic for both ingress and egress traffic. Thus, it isimportant to continually track the subscriber's ingress and egresstraffic usage across all NPUs to note if and when the quota amount isexceeded. In gateways utilizing an architecture including multipleprocessing units within the forwarding plane, tracking the subscriber'singress and egress traffic usage is difficult because of the continualneed for communication between each NPU and the control plane to reportupon the different types and amounts of traffic used.

One mechanism for splitting a received quota amount between NPUsinvolves splitting the quota amount into two portions according to apredicted usage ratio between uplink and downlink traffic and assigningeach portion to a corresponding NPU. Then, when one of the NPUs exhaustsits portion of the quota amount, the gateway will signal the remoteserver for additional quota. However, in this approach, the gatewayperforms inefficiently as it must frequently signal the external serverto request new quota and then wait for responses from the externalserver.

The embodiments of the present invention provide a method and apparatusto manage the assignment of quota amounts between NPUs in a multiple NPUarchitecture forwarding plane by intelligently allocating andreallocating quota amounts between NPUs to minimize internal controlmessaging between the control plane and the forwarding plane and tominimize signaling between the control plane and external quota-grantingservers.

Additionally, some of the following embodiments may be described asincluding a network element with two NPUs, and other embodiments may bedescribed as including a network element with more than two NPUs. Unlessindicated otherwise, the following configurations and techniques areapplicable to both types of embodiments, and a configuration ortechnique described for one of the embodiments could be adapted to workwith the other embodiment.

FIG. 1 illustrates an exemplary system including a network element 106utilizing multiple network processing units according to one embodimentof the invention. A subscriber end station 100 is communicativelycoupled with the network element 106 through a network 102, which in oneembodiment may be a 3G LTE network. In other embodiments, the network102 may be another type of wireless network (e.g. WiMAX, TDMA-based,etc.) or a wired network. The connection between the subscriber endstation 100 and network element 106 occurs to allow the subscriber endstation 100 to communicate with one or more end stations 122A-122Naccessible to the network element 106 across a data network 120. In anembodiment, the data network 120 is an external packet data network suchas the Internet.

The network element 106 contains both a control plane 108 as well as aforwarding plane 110 (also known as a data plane). The forwarding plane110 is a multiple NPU architecture forwarding plane 110 in that itincludes a first NPU 116A and a second NPU 116B. In an embodiment, oneof these NPUs 116A-116B may serve as an egress processor and the othermay serve as an ingress processor. It is also possible, in anotherembodiment, to have the first NPU 116A and the second NPU 116B eachprocess both ingress and egress traffic. Further, in some embodiments,the forwarding plane 110 may have more than two NPUs (e.g. 116A-116N),where an NPU may be a separate directional forwarding processor(exclusively processing egress traffic, or exclusively processingingress traffic) or a dual-directional forwarding processor (processingboth egress and ingress traffic).

The subscriber end station 100 transmits a service data request to thenetwork element 106 through the network 102. The service data request isa request to send data to or receive data from an end station (e.g.122A) using the data network 120. If this request from the subscriberend station 100 is the first request requiring connectivity to the datanetwork 120, the network element 106 creates a subscriber session. Inanother embodiment, at the time a subscriber end station 100 connects tothe network 102, before any request for data network 120 connectivity,the network element 106 creates a subscriber session. At or after thetime the subscriber session is created on the network element 106, aremote quota server 104 assigns the subscriber session a quota amount130 for one or more traffic flows via a quota provision message 158.This provisioned quota amount 130 is an amount of data for a particularflow that can be sent or received through the network element 106 by thesubscriber end station 100. The quota amount 130 is stored by a quotatracking module 114, which maintains various statistics regardingsubscriber flows.

Upon receipt of the quota provision message 158 and determination of thequota amount 130 for a flow, the quota management module 112 performs aninitial quota assignment 150 to a first NPU (e.g. 116A). This initialquota assignment 150 contains a quota amount 119A, which is a portion ofthe quota amount 130 signifying an amount of traffic that the first NPU116A may utilize for a particular subscriber flow. This quota amount119A is stored by the first NPU 116A as part of a subscriber flow quota118A. Similarly, this quota assignment also occurs for other NPUs withinthe forwarding plane 110. For example, the quota management module 112also makes an initial quota assignment 151 to a second NPU (e.g. 116B),which stores the portion of the quota amount 130 communicated in theinitial quota assignment 151 as a subscriber flow quota 118B quotaamount 119B.

In this manner, the network element 106 is enabled to perform thenetwork forwarding necessary for the service data request generated bythe subscriber end station 100. As an example, when the first NPU 116Ais configured as a dedicated ingress NPU to process uplink traffic, thefirst NPU 116A will forward the service data request to the data network120. In forwarding this request, the first NPU 116A will note the amountof data forwarded and increment the consumed amount 121A of thesubscriber flow quota 118A that the service data request is associatedwith. Thus, the first NPU 116A maintains a record of the assigned quotaamount 119A and the consumed amount 121A of the subscriber flow quota118A. Similarly, with the second NPU 116B configured as a dedicatedegress NPU to process downlink traffic, when the network element 106receives the service data request's return traffic data from an endstation (e.g. 122A), the second NPU 116B will forward the data to thesubscriber end station 100 using the network 102 and increment aconsumed amount 121B of the subscriber flow quota 118B according to theamount of return traffic data.

When the network element 106 receives a quota amount 130 as part of aquota provision message 158 from a remote quota server 104, the quotamanagement module 112 must determine how to initially distribute thequota amount 130 between the first NPU 116A and second NPU 116B. One wayto split the data is according to a defined ratio set in the quotamanagement module 112. In one embodiment, this defined ratio is basedupon a common case illustrated by a user traffic model: frequent trafficusage observed in 3G LTE networks is approximately 20% uplink trafficand 80% downlink traffic. Thus, an initial split using this user model20:80 ratio will assign 20% of the quota amount 130 to ingress NPUs and80% of the quota amount 130 to egress NPUs.

For example, the network element 106 of FIG. 1 illustrates an initialquota assignment according to the 20:80 defined ratio with one ingressNPU 116A and one egress NPU 116B. In this example, the remote quotaserver 104 assigns the network element 106 a total of one gigabyte (1GB) of quota amount 130 for a particular subscriber flow via a quotaprovision message 158. Using the 20:80 defined ratio, the quotamanagement module 112 assigns the first NPU 116A a subscriber flow quota118A quota amount 119A of two hundred megabytes (200 MB) of traffic,which is 20% of the 1 GB quota amount 130. Similarly, using the 20:80defined ratio, the quota management module 112 assigns the second NPU116B a subscriber flow quota 118B amount 119B of eight hundred megabytes(800 MB) of traffic, which is 80% of the 1 GB quota amount 130.

In another embodiment, the defined ratio used to split the quota amount130 is based upon a historic NPU usage ratio. The historic NPU usageratio indicates a ratio of ingress traffic to egress traffic consumedduring previous communications between the network element 106 and thesubscriber end station 100. This ratio may further be limited toincluding only traffic from communications for the same subscriber flowthat the quota amount to be apportioned is associated with. For example,if the quota amount 130 to be split is for a flow of VoIP traffic, theratio may be based upon the user's previous amount of VoIP ingress andegress traffic usage. A benefit of using defined ratios that approximateanticipated levels of ingress and egress usage for a flow when splittingthe quota amount 130 is increased system efficiency. In a forwardingplane 110 with two NPUs (e.g. 116A-116B), if each NPU is assigned aportion of the quota amount in proportion to the amount each will likelyuse in a given time period, it is likely that both of the NPUs willconsume their assigned quota amounts at approximately the same time.This situation is preferred to a scenario where a first NPU (e.g. 116A)consumes its quota amount far before a second NPU (e.g. 116B). In thisscenario, the first NPU 116A must be replenished with additional quota,which requires additional signaling and action on the part of the quotamanagement module 112. However, if the quota amount was divided betweenthe two NPUs in a proportion more closely linked to the actual usageproportion, the NPUs would be able to operate longer without running outof quota. Additionally, if the initial quota assignment is more closelyaligned with the actual usage proportion, there will only be oneexchange between the NPUs and the quota management module for a singlequota, which is more efficient and may allow the system to scale tohandle more subscribers.

As the network element 106 continues to serve the subscriber end station100 by providing access to the data network 120, one of the NPUs (e.g.116A) will consume a significant portion of its subscriber flow quotaamount 119A. At this point, the NPU 116A will signal the quotamanagement module 112 using a quota exhausted message, and may reset itsquota consumed amount 121A back to 0. In an embodiment, the significantportion of the quota amount 119A is the entirety of the quota amount119A, but other embodiments may cause a quota exhausted message to besent earlier, such as when a particular percentage (e.g. 75%, 90%, etc.)of the quota amount 119A is consumed. In an embodiment, the quotaexhausted message will include a value representing the portion of thequota amount that has been consumed. For example, if NPU 116A were tosend a quota exhausted message in such an embodiment, the quotaexhausted message would include the consumed amount 121A of thesubscriber flow quota 118A (e.g. 150 MB, which is 75% of the quotaamount 119A of 200 MB).

Upon receipt of a quota exhausted message from a first NPU 116A, thequota management module 112 determines how much of the quota amount 130is remaining, or unconsumed, within the NPUs in the forwarding plane110. The quota management module 112 assigns a portion of thisunconsumed quota amount to a first NPU 116A through a dynamic quotaupdate 152, and assigns another portion of this unconsumed quota amountto the second NPU 116B through another dynamic quota update 153.

Additionally, upon receipt of a quota exhausted message from an NPU, thequota management module 112 may signal a remote quota server 104. In anembodiment, the quota management module 112 sends a quota consumedreport message 157 to the remote quota server 104 to report the quotaconsumed amount 132, which is an amount of the quota amount 130 that hasbeen consumed by the NPUs. Upon receipt of a quota consumed reportmessage 157, the remote quota server 104 may be configured to grant anadditional quota amount for the subscriber flow to the network element106 with a quota provision message 158, provided that the subscriber isallowed to consume additional quota. In another embodiment, the quotamanagement module 112 may send a subscriber quota request message to theremote quota server 104 to explicitly ask for an additional quota amountfor the subscriber flow.

The quota management module 112 may maintain information aboutsubscriber flows in an optional quota tracking module 114. This quotatracking module 114 may be accessed by the quota management module 112via a read value request, and the quota tracking module 114 will sendback a requested value with a return value message 155. The quotamanagement module 112 is also operable to update a value in the quotatracking module 114 using an update value 156 message.

The determination by the quota management module 112 regarding how tosplit the remaining quota amount between the NPUs may occur through theuse of the information maintained by the quota tracking module 114. Forexample, when a received quota provision message 158 assigns an amountof quota for a subscriber flow, the quota management module 112 cachesthis value as a quota amount 130 in the quota tracking module 114. Whenan NPU signals the quota management module 112 with a quota exhaustedmessage and the quota management module 112 calculates an unconsumedquota amount, the quota management module 112 further calculates andstores a quota consumed amount 132 within the quota tracking module 114.Similarly, the quota management module 112 may also note the portions ofthe quota amount that each NPU in the forwarding plane 110 has consumedin relation to the portions consumed by the other one or more NPUs, andstore representations of these as NPU usage ratios 131 within the quotatracking module 114.

For example, when a subscriber end station 100 first becomescommunicatively coupled with the network element 106, the remote quotaserver 104 assigns the subscriber session a total of 1 GB of trafficdata through a quota provision message 158. The quota management module112 records this 1 GB within the quota tracking module 114 as a quotaamount 130 and makes an initial quota assignment 150 (e.g. 200 MB) to afirst NPU 116A and an initial quota assignment 151 (e.g. 800 MB) to asecond NPU 116B. As the network element 106 processes the service datarequest and subsequent service data requests, quota will be consumed bythe first NPU 116A and/or the second NPU 116B. At some point, accordingto configured settings specifying when an NPU should send a quotaexhausted message, the second NPU 116B sends a quota exhausted messageto the quota management module 112 indicating that of its assigned 800MB of quota amount 119B, its quota consumed 121B amount is 650 MB. Atthis point, the second NPU 116B may also reset its quota consumed amount121B back to 0. In response, the quota management module 112 queries thefirst NPU 116A to determine how much of its assigned 200 MB of quotaamount 119A that it consumed. The first NPU 116A will report that itconsumed 150 MB as its quota consumed amount 121A. With these values,the quota management module 112 determines that of the 1 GB of quotaamount 130 for this subscriber flow, 800 MB has been consumed, whichwill be stored as the quota consumed amount 132, and that 200 MB is anunconsumed quota amount. Further, the quota management module 112 maycalculate that the first NPU 116A has consumed 18.75% of the quotaconsumed amount 132, and store a representation of this percentage as anNPU usage ratio 131. Similarly, the quota management module 112 maycalculate that the second NPU 116B has consumed 81.25% of the quotaconsumed amount 132, and also store a representation of this percentageas an NPU usage ratio 131.

When determining how to redistribute unconsumed quota, the quotamanagement module 112 may utilize similar splitting methods as thoseused during the initial quota assignment. For example, the quotamanagement module 112 may split the unconsumed quota amount using adefined ratio (e.g. according to a user traffic model, 20:80, etc.) orusing a historic NPU usage ratio.

Alternatively, the quota management module 112 may use the NPU usageratios 131 tracked in the quota tracking module 114 to determine whatportions of the unconsumed quota amount will be redistributed to each ofthe NPUs 116A-116B. Continuing the example with 200 MB of unconsumedquota, the quota management module 112 will assign the first NPU 116A aportion according to the first NPU's 116A NPU usage ratio 131 (e.g.18.75%). Thus, the quota management module 112 will assign the first NPU116A a quota amount 119A of 37.5 MB (18.75% of 200 MB) through a dynamicquota update 152. Similarly, the quota management module 112 will assignthe second NPU 116B a quota amount 119B of 162.5 MB (81.25% of 200 MB)through a dynamic quota update 153. This method of redistributing quotabetween NPUs is very efficient as each NPU is granted a portion ofunconsumed quota in proportion to how much it has recently used comparedto the one or more other NPUs in the forwarding plane. By assuming thefuture need for ingress and egress traffic will be similar to the recentneed for ingress and egress traffic as indicated by the NPU usage ratios131, the system is able to make an empirical estimation of the ratio ofquota that should be assigned to the NPUs to allow each to continueprocessing data for the flow as long as possible before any NPU willexhaust its assigned quota amount.

In a system having exactly two NPUs 116A-116B, the quota tracking module114 may be configured to only store one NPU usage ratio 131 for one ofthe NPUs, as the other can be easily derived. For example, if the firstNPU 116A consumes 30% of the quota amount in a two NPU system, only thisvalue needs to be stored, as the ratio for the second NPU 116B may bederived using simple subtraction to be 70% (i.e. 100−30=70). However, ina system with more than two NPUs, the quota tracking module 112 wouldneed to keep representations of the ratio for each NPU to be able toredistribute quota using the NPU usage ratios 131 in this manner.Alternatively, in an embodiment, the NPU usage ratios 131 track only oneratio per flow, which is a ratio of ingress to egress traffic. In thiscase, the quota management module 112 would assign a portion of theingress portion of the unconsumed quota amount to each ingress NPU inthe forwarding plane 110, and similarly assign a portion of the egressportion of the unconsumed quota amount to each egress NPU in theforwarding plane 110.

The quota tracking module 114 may also be operable to store a requestthreshold 133, which can be used by the quota management module 112 indetermining when to signal the remote quota server 104. This requestthreshold 133 may be associated for all subscriber flows on the networkelement 106 or may be associated with one or more subscriber flows, andrepresents a percentage of a provisioned quota amount 130 that, whenconsumed by the NPUs, indicates a need to signal the remote quota server104. Thus, when the quota management module 112 receives a quotaexhausted message from an NPU and updates the quota consumed amount 132for the flow, it will determine if the quota consumed amount 132 dividedby the quota amount 130 (which is a quota consumed percentage value) isgreater than the request threshold 133. If so, the quota managementmodule 112 will signal the remote quota server 104. In anotherembodiment, the request threshold 133 is not a ratio of quotaconsumption but is instead an amount of data (e.g. in bytes, kilobytes,etc.) to be consumed. Thus, when the quota consumed amount 132 firstequals or exceeds the request threshold 133, the quota management module112 will signal the remote quota server 104.

Similarly, in an embodiment the quota tracking module 114 is operable tomanage a redistribution count 134 and a redistribution limit 135. In anembodiment, the redistribution limit 135 is defined for all subscriberflows managed by the network element 106. This redistribution limit 135represents a number of times that unconsumed quota should beredistributed between the NPUs since the provisioned quota amount 130was assigned in a quota provision message 158 before signaling theremote quota server 104. Similarly, the redistribution count 134 keepstrack of the number of times that unconsumed quota has beenredistributed between the NPUs since the quota amount 130 was originallyassigned in a quota provision message 158. In an embodiment, when theredistribution count 134 equals the redistribution limit 135, the quotamanagement module 112 signals the remote quota server 104.

In an embodiment, a quota management module 112 is operable to implementboth a request threshold 133 and a redistribution limit 135simultaneously. In such a configuration, when either the quota consumedpercentage value exceeds the request threshold 133 or the redistributioncount 134 equals the redistribution limit 135, the quota managementmodule 112 will signal the remote quota server 104.

In an embodiment, when the quota management module 112 signals theremote quota server 104 the quota management module 112 will no longerredistribute quota upon receipt of a quota exhausted message from anyNPU until an additional provisioned quota amount is assigned by theremote quota server 104 in a quota provision message 158. Thisconfiguration provides a benefit of reduced internal signaling betweenthe forwarding plane 110 and the control plane 108. In an embodiment,each NPU will be made aware of the signal to the remote quota server 104and will not send quota exhausted messages until the NPUs are assignedadditional quota when the quota management module 112 receives anadditional provisioned quota amount. Similarly, this embodiment alsoprovides a benefit of eliminating signaling between the forwarding plane110 and the control plane 108 regarding the quota while the systemawaits additional quota.

To illustrate this benefit, consider the alternate scenario where thequota management module 112 would always continue to redistribute quotato two NPUs 116A-116B. After some time redistributing unconsumed quotaamounts, the quota management module 112 would calculate a very smallunconsumed quota amount and split this very small amount into two evensmaller portions. The first small portion would be assigned to a firstNPU 116A and the second small portion would be assigned to a second NPU116B. In a very short period of time, one of the NPUs (e.g. 116A) wouldconsume its assigned first small portion of quota very quickly and senda quota exhausted message to the quota management module 112. This wouldtrigger the quota management module 112 to signal the second NPU 116B,receive a response from the second NPU 116B, and use the response tocalculate an even smaller unconsumed quota amount. This unconsumed quotaamount would again be split into even smaller portions for the NPUs116A-116B. With the NPUs 116A-116B receiving smaller and smallerportions of quota, the exhaustion-redistribution cycle would intensifyin frequency, potentially leading to an overwhelming amount of signalingbetween the control plane 108 and the forwarding plane 110.

In an embodiment, when the quota management module 112 has signaled theremote quota server 104 and refuses to redistribute unconsumed quotabecause the remote quota server 104 has not yet sent an additionalprovisioned quota amount, the quota management module 112 may allow eachNPU that exhausts its assigned quota amount to continue processing extratraffic for the flow, and then deduct this extra traffic usage from anyfuture provisioned quota amount received from the remote quota server104. In this configuration, the subscriber will not notice anyinterruption of service caused by an NPU pausing the processing oftraffic while it waits for additional quota. However, if the remotequota server 104 determines that the subscriber flow is not allowedadditional quota and signals the forwarding element to stop providingservice, the subscriber may receive the benefit of some “free” servicebefore such a message is received. In another embodiment, instead ofcontinuing processing, the NPUs may be configured to stop processing anytraffic. In this approach, the subscriber may notice an interruption ofservice, but the subscriber is prevented from ever gaining any “free”service.

The operations of the flow diagrams in FIGS. 2-5 and 7 will be describedwith reference to the exemplary embodiment of FIG. 1. However, it shouldbe understood that the operations of flow diagrams can be performed byembodiments of the invention other than those discussed with referenceto FIG. 1, and the embodiments discussed with reference to FIG. 1 canperform operations different than those discussed with reference to theflow diagrams of FIGS. 2-5 and 7.

FIG. 2 illustrates a flow diagram of a method in a network element forassigning and redistributing quota between a set of network processingunits for a subscriber flow according to an embodiment of the invention.In block 206, the quota management module 112 determines a quota amount130. In an embodiment, a provisioned quota amount 130 arrives as part ofa quota provision message 158 from a remote quota server 104. In block208, the quota amount 130 is split into a first quota portion and asecond quota portion. In an embodiment, this split occurs according to adefined ratio; in other embodiments, this split occurs according to ahistoric NPU usage ratio indicating a ratio of ingress traffic to egresstraffic consumed during all previous communications between the networkelement 106 and the subscriber end station 100 for a particularsubscriber flow. In block 210, the first quota portion is assigned to afirst NPU (e.g. 116A), and in block 212 the second quota portion isassigned to a set of one or more other NPUs (e.g. 116B-116N). In anembodiment where the forwarding plane 110 includes exactly two NPUs(e.g. 116A-116B), the second quota portion is simply the differencebetween the first quota portion and the quota amount 130. In anotherembodiment where the forwarding plane 110 has more than two NPUs (e.g.116A-116N), the second quota portion may further be split into two ormore portions, each of which to be assigned to an NPU. In block 214, thequota management module 112 is to determine to change the distributionof an unconsumed quota amount. In an embodiment of the invention, thedetermination occurs as a result of the quota management module 112 inthe control plane 108 receiving a quota exhausted message from an NPU(e.g. 116A) in the forwarding plane. In block 216, the quota managementmodule 112 determines an amount of unconsumed quota. In an embodiment,the quota management module 112 sends a quota consumed query message toeach NPU (e.g. 116B-116N) that did not send the quota exhausted message,causing the quota management module 112 to determine to change thedistribution of an unconsumed quota amount. In response to receipt ofthe quota consumed query message, each recipient NPU (116B-116N) sends anotification to the quota management module 112 indicating how muchquota it consumed or did not consume. With this information, the quotamanagement module 112 is then able to calculate how much of the quotaamount 130 remains unconsumed. In block 218, the quota management module112 assigns a first portion of the unconsumed quota amount to a firstNPU 116A. In one embodiment, the quota management module 112 determinesa first portion of the unconsumed quota amount using an NPU usage ratio131 for the first NPU 116A, where the NPU usage ratio 131 indicates whatpercentage of quota for the subscriber flow has been consumed by thefirst NPU 116A. The first portion of the unconsumed quota amount iscalculated to be the percentage of the unconsumed quota amount indicatedby the first NPU's 116A corresponding NPU usage ratio 131. In block 220,the quota management module 112 assigns a second portion of theunconsumed quota amount to the set of other NPUs 116B-116N. In anembodiment with exactly two NPUs in the forwarding plane 110, the secondportion of the unconsumed quota amount is calculated to be thedifference between the unconsumed quota amount and the first portion ofthe unconsumed quota amount. In another embodiment with more than twoNPUs in the forwarding plane 110, the quota management module 112assigns the second portion of the unconsumed quota amount by dividing aportion of the unconsumed quota amount into smaller portions andassigning these smaller portions to the remaining NPUs 116B-116N. Thismay occur by calculating an NPU-specific portion of the unconsumed quotaamount for each NPU in the set of other NPUs 116B-116N according to eachNPU's usage ratio 131, and then assigning each NPU-specific portion tothe corresponding NPU.

FIG. 3 illustrates a flow diagram of a method in a network element fordetermining an unconsumed quota amount according to an embodiment of theinvention. Block 302 represents one embodiment of block 214, whichencompasses the step of determining to change the distribution of anunconsumed quota amount. In this embodiment, the determination occurswhen the quota management module 112 in the network element 106 receivesa quota exhausted message from a first NPU (e.g. 116A) in the forwardingplane 110. In an embodiment, the quota exhausted message indicates thatthe first NPU 116A has consumed its entire portion of assigned quota,but in other embodiments the quota exhausted message may simply indicatethat the first NPU 116A has consumed an amount of its assigned quota.

Blocks 304, 306, and 308 represent an embodiment of block 216, whichencompasses the step of determining an amount of unconsumed quota for asubscriber flow. In block 304, the quota management module 112 sends aquota consumed query message to a set of one or more other NPUs (e.g.116B-116N), where each message indicates a request for each NPU toreport how much of their assigned quota for the flow they have consumed.In block 306, the quota management module 112 receives quota consumedreport messages from the set of NPUs 116B-116N. In an embodiment, uponreceipt of a quota consumed query message from the quota managementmodule 112, an NPU sends a quota consumed report message back to thequota management module 112 including the NPU's consumed amount of itsassigned portion of the subscriber flow quota amount. Thus, aftersending quota consumed query messages to the set of NPUs 116B-116N inblock 304, each NPU in the set will respond to the quota managementmodule 112 with a quota consumed report message. In block 308, the quotamanagement module 112 determines the unconsumed quota amount based atleast on the received quota consumed report messages in block 306. In anembodiment, the quota management module 112 determines the unconsumedquota amount by calculating the difference between the quota amount 130and a quota consumed amount 132, which is calculated using all consumedamounts from the received quota consumed query messages and the consumedamount indicated by the quota exhausted message from the first NPU 116A.

In block 310, the quota management module 112 assigns a portion of theunconsumed quota amount to the first NPU 116A according to the firstNPU's usage ratio 131. In an embodiment, the quota management module 112maintains NPU usage ratios 131 for each NPU in the forwarding plane 110using the quota tracking module 114, each NPU usage ratio 131 indicatingthe portion of the quota amount 130 that the corresponding NPU hasconsumed. The quota management module 112 will perform a dynamic quotaupdate 152 of the first NPU 116A by assigning it a portion of theunconsumed quota amount in proportion to the NPU usage ratio 131 for thefirst NPU 116A. In block 312, the quota management module 112 assignsone or more portions of the unconsumed quota amount to the set of NPUs116B-116N according to the NPU usage ratio 131 for each NPU in the set.In an embodiment where the forwarding plane 110 contains exactly twoNPUs (e.g. 116A-116B), the NPU usage ratio 131 for the second NPU 116Bmay be explicitly stored or derived, as the second NPU usage ratio 131is equal to the difference between the number one and the first NPU's116A NPU usage ratio 131. For example, if the first NPU's 116A NPU usageratio 131 is 0.55, then the second NPU's 116B NPU usage ratio 131 may bederived as equal to one minus 0.55, or 0.45. In an embodiment, an NPU inthe set of NPUs 116B-116N is assigned a quota amount based upon theunconsumed quota amount multiplied by the corresponding NPU's NPU usageratio 131.

FIG. 4 illustrates a flow diagram of a method in a network element fordetermining an unconsumed quota amount according to an embodiment of theinvention. The flow of FIG. 4 differs from the flow of FIG. 3 at leastin that it determines an amount of unconsumed quota for a subscriberflow differently. Block 402 represents one embodiment of block 214, asthe quota management module 112 determines to change the distribution ofan unconsumed quota amount by receiving a quota exhausted message from afirst NPU (e.g. 116A).

Blocks 404, 406, and 408 represent an embodiment of block 216, whichencompasses the step of determining an amount of unconsumed quota for asubscriber flow. In block 404, the quota management module 112 sends aquota available query message to a set of one or more other NPUs (e.g.116B-116N), which indicates a request for each NPU 116B-116N to reporthow much of their assigned quota for the flow that has not beenconsumed. In block 406, the quota management module 112 receives quotaavailable report messages from the set of NPUs 116B-116N. In anembodiment, upon receipt of a quota available query message from thequota management module 112, an NPU sends a quota available reportmessage back to the quota management module 112 including an availableamount, which is the NPU's unconsumed amount of its assigned portion ofthe subscriber flow quota amount. In one embodiment, an NPU (e.g. 116B)determines this unconsumed amount of its assigned portion of thesubscriber flow quota amount by calculating the difference between itsassigned quota amount 119B and its consumed amount 121B. After sendingquota available query messages to the set of other NPUs 116B-116N inblock 404, each NPU in the set will respond to the quota managementmodule 112 with a quota available report message. In block 408, thequota management module 112 determines the unconsumed quota amount basedon the received quota available report messages in block 306. In anembodiment where a quota exhausted message indicates a NPU (e.g. 116A)has consumed its entire assigned quota amount, the quota managementmodule 112 determines the unconsumed quota amount by calculating the sumof all available amounts (e.g. 119B-119N) from the received quotaavailable query messages. In an embodiment where a quota exhaustedmessage does not mean the NPU (e.g. 116A) has consumed its entireassigned quota amount, the quota management module 112 determines theunconsumed quota amount by calculating the sum of all available amounts(e.g. 119B-119N) from the received quota available query messages andfurther adding an available amount from the first NPU 116A. In oneembodiment, the quota exhausted message from the first NPU 116A includesan available amount, which is the difference between the NPU's assignedquota amount 119A and the amount of that quota that has been consumed121A. In another embodiment, the quota exhausted message from the firstNPU 116A includes a consumed amount 121A, and the quota managementmodule 112 must calculate an available amount for the first NPU 116A bysubtracting the consumed amount 121A from the quota amount 119A assignedto the first NPU 116A. In another embodiment, the quota managementmodule 112 may determine the available amount from the first NPU 116A byfurther sending a quota available query message to the first NPU 116Aand receiving a quota available report message back from the first NPU116A that includes an available amount.

In block 410, the quota management module 112 assigns a portion of theunconsumed quota amount to the first NPU 116A according to the firstNPU's usage ratio 131. In an embodiment, the quota management module 112maintains NPU usage ratios 131 for each NPU in the forwarding plane 110using the quota tracking module 114, each NPU usage ratio 131 indicatingthe portion of the quota amount 130 that the NPU has consumed. The quotamanagement module 112 will perform a dynamic quota update 152 of thefirst NPU 116A by assigning it a portion of the unconsumed quota amountin proportion to the NPU usage ratio 131 for the first NPU 116A. Inblock 412, the quota management module 112 assigns one or more portionsof the unconsumed quota amount to the set of NPUs 116B-116N according tothe NPU usage ratio 131 for each NPU in the set. In an embodiment wherethe forwarding plane 110 contains exactly two NPUs (e.g. 116A-116B), theNPU usage ratio 131 for the second NPU 116B may be explicitly stored orderived, as the second NPU usage ratio 131 is equal to the differencebetween the number one and the first NPU's 116A NPU usage ratio 131. Forexample, if the first NPU's 116A NPU usage ratio 131 is 0.55, then thesecond NPU's 116B NPU usage ratio 131 may be derived as equal to oneminus 0.55, or 0.45. In an embodiment, an NPU in the set of NPUs116B-116N is assigned a quota amount based upon the unconsumed quotaamount multiplied by the NPU's corresponding NPU usage ratio 131.

FIG. 5 illustrates a flow diagram of a method in a network element forquota redistribution according to an embodiment of the invention. Inblock 502, the quota management module 112 receives a quota exhaustedmessage from a first NPU (e.g. 116A).

In response to the receipt of this quota exhausted report message, thequota management module 112 sends one or more quota query messages toeach of a set of one or more other NPUs (e.g. 116B-116N) in theforwarding plane 110 as in block 504. In block 506, the quota managementmodule 112 receives one or more quota report messages from the set ofNPUs 116B-116N. In one embodiment of the invention, the one or morequota query messages are quota consumed query messages and the one ormore quota report messages are quota consumed report messages. Inanother embodiment of the invention, the one or more quota querymessages are quota available query messages and the quota reportmessages are quota available report messages.

In block 508, the quota management module 112 determines a quotaconsumed amount based on the contents of the quota report messages. Adecision point is presented in block 510, wherein the quota managementmodule 112 determines if a quota consumed percentage, which is the quotaconsumed amount 132 divided by the quota amount 130, is greater than aquota request threshold 133. If the quota consumed percentage is greaterthan the quota request threshold 133, the flow continues to block 512and the quota management module 112 sends a quota consumed reportmessage to a remote quota server 104. In one embodiment, the quotamanagement module 112 will continue to block 520; in other embodiments,the quota management module 112 is configured to stop redistributingunconsumed quota until an additional provisioned quota amount isreceived from the remote quota server 104.

When decision block 510 evaluates to “N,” or in embodiments where block512 is to be followed by block 520, the quota management module 112determines an unconsumed quota amount, per block 520. The quotamanagement module 112 assigns a portion of the unconsumed quota amountto the first NPU 116A according to the first NPU's 116A correspondingNPU usage ratio 131, as described in block 522. In block 524, the quotamanagement module 112 assigns one or more portions of the unconsumedquota amount to the set of other NPUs 116B-116N according to each NPU'scorresponding usage ratio 131.

FIG. 6 illustrates a similar flow diagram of a method in a networkelement for quota redistribution according to an embodiment of theinvention, and differs from FIG. 5 at least in that it utilizes adifferent method for determining whether to send a quota consumed reportmessage to a remote quota server. In block 602, the quota managementmodule 112 receives a quota exhausted message from a first NPU (e.g.116A). The quota management module 112 determines to redistribute anunconsumed quota amount, and sends one or more quota query messages to aset of one or more other NPUs 116B-116N in block 604. The quotamanagement module 112 receives one or more quota report messages fromthe set of other NPUs 116B-116N in block 604. In one embodiment of theinvention, the one or more quota query messages are quota consumed querymessages and the one or more quota report messages are quota consumedreport messages. In another embodiment of the invention, the one or morequota query messages are quota available query messages and the one ormore quota report messages are quota available report messages.

In block 608, the quota management module 112 increments aredistribution count 134, and then a decision point occurs in block 610:if the redistribution count 134 is equal to a redistribution limit 135,the quota management module 112 sends a quota consumed report message toa remote quota server 104 in block 612. Depending upon the embodiment ofthe invention, the quota management module 112 may or may not refuse toredistribute any remaining unconsumed quota amount. If theredistribution count 134 is not equal to a redistribution limit 135, thequota management module 112 will determine an unconsumed quota amount asin block 620. In block 622, the quota management module 112 will assigna portion of the unconsumed quota amount to the first NPU 116A accordingto a corresponding NPU usage ratio 131 for that NPU 116A, and willassign one or more portions of the unconsumed quota amount to each ofthe set of other NPUs 116B-116N according to each NPU's correspondingNPU usage ratio 131.

An example illustrating the flow of FIG. 5 is presented in FIG. 7, whichillustrates a sequence diagram depicting a quota management process in anetwork according to an embodiment of the invention. Similarly, anexample illustrating the flow of FIG. 6 is represented in FIG. 8, whichalso illustrates a sequence diagram depicting a quota management processin a network according to an embodiment of the invention. FIGS. 7 and 8differ in how to determine whether to send a quota consumed reportmessage to a remote quota server, and these differences will be detailedin the following paragraphs.

In one embodiment, a remote quota server 104 sends a quota provisionmessage 712 to the network element 106 with a subscriber flowprovisioned quota amount of 1 GB of traffic quota. The quota managementmodule 112 notes the 1 GB of quota amount 130 for the subscriber flowand assigns a first portion of the quota amount—200 MB—to a first NPU116A via a quota provision message 714. The quota management module 112also assigns a second portion of the quota amount—800 MB—to a second NPU116B via a quota provision message 716. In this embodiment, the quotamanagement module 112 determined the first portion of the quota amountand the second portion of the quota amount using a defined ratio of20:80. In an embodiment where the forwarding plane 110 contains morethan two NPUs (e.g. 116A-116N), this 800 MB may be assigned to a set ofone or more other NPUs 116B-116N using multiple quota provisionmessages. After processing data for the subscriber flow for some periodof time, the first NPU 116A consumes an amount of quota and sends thequota management module 112 a quota exhausted message 718. In thisembodiment, the first NPU 116A has consumed the entire portion of itsassigned quota—200 MB—but in other embodiments, the quota exhaustedmessage 718 may indicate that the first NPU 116A has consumed an amountof quota 121A that is less than the entire assigned quota amount 119A.

Upon receipt of the quota exhausted message 718, the quota managementmodule 112 determines to change the distribution of the unconsumed quotaamount. To determine the unconsumed quota amount, the quota managementmodule 112 sends a quota consumed query message 720 to NPU 116B. Inresponse, NPU 116B sends a quota consumed report message 722 to thequota management module 112 that indicates it has consumed 200 MB of itsassigned 800 MB.

The quota management module 112 calculates a quota consumed amount 723of 400 MB based upon the quota exhausted message 718 and the quotaconsumed message 722. In an embodiment with more than two NPUs (e.g.116A-116N), the quota management module 112 sends a quota consumed querymessage to all NPUs 116B-116N, receives quota consumed messages fromNPUs 116B-116N, and calculates a quota consumed amount based upon thequota exhausted message 718 from the first NPU 116A and all quotaconsumed report messages received from the set of other NPUs 116B-116Nin step 722. In this scenario the quota consumed percentage, which isbased on the quota consumed amount 723, is not greater than a quotarequest threshold 133, so the quota management module 112 will notsignal the remote quota server 104 at this time. Having determined thatthe quota consumed amount is 400 MB, the quota management module 112calculates that there is 600 MB of unconsumed quota amount remaining tobe redistributed.

Instead of calculating a quota consumed amount 723, the embodiment ofFIG. 8 presents a method of operation where the quota management module112 instead increments a redistribution count 823 to track the number oftimes the provisioned quota amount 130 has been redistributed betweenthe NPUs 116A-116N in the forwarding plane 110. Because theredistribution count 823 does not equal a defined redistribution limit135, the quota management module 112 will not signal the remote quotaserver 104 at this time. In this embodiment, the quota management module112 will calculate that there is 600 MB of unconsumed quota amountremaining to be redistributed based upon the quota exhausted message 718and all received quota consumed report messages 722. In anotherembodiment, the quota management module 112 may both increment aredistribution count 823 as well as calculate a quota consumed amount723.

Turning back to FIG. 7, the quota management module 112 updates an NPUusage ratio 131 corresponding to the first NPU 116A, noting that it hasconsumed 50% of the quota consumed amount 132, which is 200 MB of the400 MB. The quota management module 112 also updates an NPU usage ratio131 corresponding to the second NPU 116B, noting that it has alsoconsumed 50% of the quota consumed amount 132, which also is 200 MB ofthe 400 MB. Using these NPU usage ratios, the quota management module112 determines that 50% of the unconsumed quota amount, or 300 MB of the600 MB, shall be assigned to the first NPU 116A, and similarly 50% ofthe unconsumed quota amount, or 300 MB of the 600 MB, shall be assignedto the second NPU 116B. Thus, the quota management module 112 assigns300 MB to the first NPU 116A with a quota provision message 724 and 300MB to the second NPU 116B with a quota provision message 726. Again, inanother embodiment where there are more than two NPUs (e.g. 116A-116N),each NPU will have a corresponding NPU usage ratio 131 and theunconsumed quota amount will be divided into portions for each NPUaccording to these NPU usage ratios 131 and assigned to each NPU usingquota provision messages.

Now, the second NPU 116B exhausts its assigned quota amount 119B beforethe first NPU 116A exhausts its assigned quota amount 119A, and sendsthe quota management module 112 a quota exhausted message 728. Thistriggers the quota management module 112 to consider changing thedistribution of the unconsumed quota amount. The quota management module112 sends a quota consumed query message 730 to the first NPU 116A andreceives a quota consumed report message 732 back indicating the firstNPU 116A has a consumed amount 121A of 280 MB of its assigned quotaamount 119A of 300 MB.

In an embodiment, the quota management module 112 calculates in block734 that, of the original 1 GB of quota, the NPUs 116A-116B consumed 980MB. In an embodiment of the invention having more than two NPUs (e.g.116A-116N), after receiving the quota exhausted message 728 from thesecond NPU 116B, the quota management module 112 sends quota consumedquery messages to the first NPU 116A and all other NPUs (excluding thesecond NPU 116B, because it sent the quota exhausted message 728),receives quota consumed report messages from these NPUs, and calculatesa quota consumed amount based on the quota exhausted message 728 fromthe second NPU 116B and all received quota consumed report messages 732.

With a quota consumed amount of 980 MB, the quota management module 112notes that the quota consumed percentage of 98% is greater than a quotarequest threshold 133. In the embodiment depicted in FIG. 1, the quotarequest threshold 133 is 95%. Because the quota consumed percentageexceeds the quota request threshold 133, the quota management module 112is to send a quota consumed report message 738 to a remote quota server104. The quota management module 112 may then receive a quota provisionmessage 740 from the remote quota server 104 containing anotherprovisioned quota amount 130, and the process may repeat.

In an embodiment, when the quota consumed percentage exceeds the quotarequest threshold 133, the quota management module 112 refuses toredistribute unconsumed quota amounts between the NPUs (e.g. 116A-116N)until another quota provision message 158 containing an additionalprovisioned quota amount is received from a remote quota server 104.This configuration eliminates a potential flood of quota exhaustedmessage and quota provision message cycles from occurring as very smallportions of quota are redistributed, quickly consumed, and redistributedagain. This reduces signaling between the control plane 108 and theforwarding plane 110, and also prevents the quota management module 112from needing to communicate with the NPUs 116A-116N to calculate quotaconsumed amounts.

In another embodiment, as detailed in FIG. 8, instead of calculating aquota consumed amount 734 and determining if a quota consumed percentageis greater than a quota request threshold 736, the quota managementmodule 112 instead increments the redistribution count 834 to 2, whichindicates that the provisioned quota amount 130 has been redistributedbetween the NPUs (e.g. 116A-116N) two times. In this embodiment, adefined redistribution limit 135 is set to 2, which indicates that theprovisioned quota amount should be redistributed 2 times beforesignaling the remote quota server 104. The quota management module 112,upon determining that the redistribution count 134 is equal to thedefined redistribution limit 135, sends a quota consumed report message738 to a remote quota server 104 and may further receive a quotaprovision message 740 back from the remote quota server 104. In anembodiment, when the redistribution count 134 is equal to the definedredistribution limit 135, the quota management module 112 refuses toredistribute unconsumed quota amounts between the NPUs 116A-116N untilanother quota provision message 158 containing an additional provisionedquota amount is received from a remote quota server 104.

In an embodiment, the quota management module 112 is configured toutilize both approaches for determining when to signal the remote quotaserver 104: monitoring a first condition of whether the quota consumedpercentage is greater than a quota request threshold 133 and monitoringa second condition of whether a redistribution count 134 is equal to aredistribution limit 135. The quota management module 112 is configuredto signal the remote quota server 104 upon the first occurrence ofeither of the conditions, and thereafter refuses to redistributeunconsumed quota amounts until another quota provision message 158containing an additional provisioned quota amount is received from theremote quota server 104.

While the flow diagrams in the figures show a particular order ofoperations performed by certain embodiments of the invention, it shouldbe understood that such order is exemplary (e.g., alternativeembodiments may perform the operations in a different order, combinecertain operations, overlap certain operations, etc.). Additionally,while the invention has been described in terms of several embodiments,those skilled in the art will recognize that the invention is notlimited to the embodiments described, can be practiced with modificationand alteration within the spirit and scope of the appended claims. Thedescription is thus to be regarded as illustrative instead of limiting.

What is claimed is:
 1. An apparatus comprising: a network element, which is to be coupled between a data network and a subscriber end station, to act as a gateway to the data network for the subscriber end station and perform intelligent quota management of a quota that limits an amount of traffic sent through a subscriber traffic flow between the data network and the subscriber end station, the network element including: a first network processing unit (NPU) operable to communicate with the subscriber end station; a second NPU operable to communicate with the subscriber end station; a control plane, operable to communicate with the first NPU and second NPU, comprising a quota management module operable to: determine a quota amount for the quota associated with the subscriber traffic flow to be assigned to the first NPU and second NPU, assign a first portion of the quota amount to the first NPU and a second portion of the quota amount to the second NPU, determine to change the distribution of an unconsumed quota amount between the first NPU and the second NPU, determine the unconsumed quota amount, assign a portion of the unconsumed quota amount to the first NPU and another portion of the unconsumed quota amount to the second NPU, and send a quota consumed report message to a remote quota server, wherein the quota consumed report message includes a consumed quota amount, and wherein the consumed quota amount is to indicate an amount of the quota amount that has been consumed by the first NPU and the second NPU, and the control plane further comprising a quota tracking module operable to track a quota consumed amount, wherein the quota consumed amount is a portion of the quota amount that has been consumed by the first NPU and the second NPU, and wherein sending the quota consumed report message to the remote quota server is to occur in response to the quota consumed amount exceeding a request threshold, wherein the request threshold is to indicate a fraction of the quota amount that is to be consumed before sending the quota consumed report message to the remote quota server.
 2. The apparatus of claim 1, wherein the quota management module is further operable to communicate with the remote quota server, wherein the quota management module is to determine the quota amount to be assigned to the first NPU and the second NPU based upon a quota provision message received from the remote quota server, wherein the quota provision message includes a provisioned quota amount.
 3. The apparatus of claim 1, wherein a ratio of the first portion of the quota amount to the second portion of the quota amount is proportional to an historic NPU usage ratio, wherein the historic NPU usage ratio indicates a ratio of an amount of quota for the subscriber traffic flow that was consumed by the first NPU during all communications with the subscriber end station to another amount of quota for the subscriber traffic flow that was consumed by the second NPU during all communications with the subscriber end station.
 4. The apparatus of claim 1, wherein the quota management module is to determine to change the distribution of the unconsumed quota amount between the first NPU and the second NPU upon receipt of a quota exhausted message from the first NPU, and wherein the quota exhausted message signifies that the first NPU has consumed an amount of its assigned quota amount.
 5. The apparatus of claim 4, wherein the quota management module is to determine the unconsumed quota amount by: sending a quota consumed query message to the second NPU, wherein the quota consumed query message is to indicate a request for an amount of the portion of the quota amount that has been consumed by the second NPU; receiving a quota consumed report message from the second NPU, wherein the quota consumed report message comprises a consumed amount, and wherein the consumed amount is the amount of the portion of the quota amount that has been consumed by the second NPU; and determining the amount of the unconsumed quota amount based on the quota consumed report message.
 6. The apparatus of claim 4, wherein the quota management module is to determine the unconsumed quota amount by: sending a quota available query message to the second NPU, wherein the quota available query message is to indicate a request for an amount of the portion of the quota amount that has not been consumed by the second NPU; receiving a quota available report message from the second NPU, wherein the quota available report message comprises an available amount, and wherein the available amount is the amount of the portion of the quota amount that has not been consumed by the second NPU; and determining the amount of the unconsumed quota amount based on the quota available report message.
 7. A method in a network element acting as a gateway in a communications network for intelligent quota management, the method comprising: managing a first quota that is associated with a subscriber traffic flow and that limits an amount of traffic travelling between the network element and a subscriber end station in the subscriber traffic flow, the managing including: assigning a first quota portion to a first network processing unit (NPU), wherein the first quota portion is a portion of the first quota; assigning a second quota portion to a set of one or more other NPUs, wherein the second quota portion is another portion of the first quota; determining to change the distribution of an unconsumed quota amount of the first quota; determining the unconsumed quota amount of the first quota; assigning a third quota portion to the first NPU, wherein the third quota portion is a portion of the unconsumed quota amount of the first quota; and assigning a fourth quota portion to the set of other NPUs, wherein the fourth quota portion is another portion of the unconsumed quota amount of the first quota; and sending a quota consumed report message for the first quota to a remote quota server, wherein the quota consumed report message includes a consumed quota amount, wherein the consumed quota amount indicates an amount of the first quota that has been consumed by the first NPU and the set of other NPUs; wherein the sending the quota consumed report message for the first quota to the remote quota server occurs when a quota consumed amount exceeds a request threshold, wherein the quota consumed amount is a portion of the first quota that has been consumed by the first NPU and the set of other NPUs since a last quota provision message for the first quota was received from the remote quota server, wherein a quota provision message includes a quota provision amount, the quota provision amount indicating a quota amount to be added to the first quota, and wherein the request threshold indicates a fraction of the quota amount that is to be consumed before sending the quota consumed report message to the remote quota server.
 8. The method of claim 7, further comprising: receiving the quota provision message for the first quota from the remote quota server, wherein the quota provision message includes a provisioned quota amount indicating a quota amount for the first quota; and determining the first quota based on the provisioned quota amount.
 9. The method of claim 7, wherein the first quota portion and the second quota portion are assigned in proportion to an historic NPU usage ratio, wherein the historic NPU usage ratio indicates an amount of the first quota consumed by each NPU during all communications with the subscriber end station.
 10. The method of claim 7, wherein the third quota portion and the fourth quota portion are assigned in proportion to an NPU usage ratio, wherein the NPU usage ratio indicates an amount of the first quota consumed by the first NPU in relation to another amount of the first quota consumed by the set of other NPUs.
 11. The method of claim 7, wherein the determining to change the distribution of the unconsumed quota amount of the first quota comprises receiving a quota exhausted message from the first NPU, the quota exhausted message indicating that the first NPU has consumed an amount of the first quota portion.
 12. The method of claim 11, wherein the quota exhausted message indicates that the first NPU has consumed the first quota portion.
 13. The method of claim 11, wherein the determining the unconsumed quota amount of the first quota comprises: sending a quota consumed query message to each of the set of other NPUs, wherein each quota consumed query message indicates a request for an amount of the portion of the first quota that has been consumed by a corresponding one of the set of other NPUs; receiving one or more quota consumed report messages from the set of other NPUs, the quota consumed report messages comprising a consumed amount, wherein the consumed amount is the amount of the portion of the first quota consumed by a corresponding one of the set of other NPUs; and determining the amount of the first quota that is unconsumed based on the one or more quota consumed report messages.
 14. The method of claim 11, wherein the determining the unconsumed quota amount of the first quota comprises: sending a quota available query message to each of the set of other NPUs, wherein the quota available query message indicates a request for an amount of the portion of the first quota that has not been consumed by a corresponding one of the set of other NPUs; receiving one or more quota available report messages from the set of other NPUs, the quota available report messages comprising an available amount, wherein the available amount is the amount of the portion of the first quota not consumed by a corresponding one of the set of other NPUs; and determining the amount of the first quota that is unconsumed based on the one or more quota available report messages.
 15. An apparatus comprising: a network element, which is to be coupled between a data network and a subscriber end station, to act as a gateway to the data network for the subscriber end station and perform intelligent quota management of a quota that limits an amount of traffic sent through a subscriber traffic flow between the data network and the subscriber end station, the network element including: a first network processing unit (NPU) operable to communicate with the subscriber end station; a second NPU operable to communicate with the subscriber end station; a control plane, operable to communicate with the first NPU and second NPU, comprising a quota management module operable to: determine a quota amount for the quota associated with the subscriber traffic flow to be assigned to the first NPU and second NPU, assign a first portion of the quota amount to the first NPU and a second portion of the quota amount to the second NPU, determine to change the distribution of an unconsumed quota amount between the first NPU and the second NPU, determine the unconsumed quota amount, assign a portion of the unconsumed quota amount to the first NPU and another portion of the unconsumed quota amount to the second NPU, and send a quota consumed report message to a remote quota server, wherein the quota consumed report message includes a consumed quota amount, and wherein the consumed quota amount is to indicate an amount of the quota amount that has been consumed by the first NPU and the second NPU; and the control plane further comprising a quota tracking module operable to track a redistribution count, wherein the redistribution count is the number of times the quota amount has been redistributed between the first NPU and the second NPU, wherein sending the quota consumed report message to the remote quota server occurs in response to the redistribution count being equal to a redistribution limit, wherein the redistribution limit is to indicate the number of times the quota amount is to be redistributed before sending the quota consumed report message to the remote quota server.
 16. The apparatus of claim 15, wherein the quota management module is further operable to communicate with the remote quota server, wherein the quota management module is to determine the quota amount to be assigned to the first NPU and the second NPU based upon a quota provision message received from the remote quota server, wherein the quota provision message includes a provisioned quota amount.
 17. The apparatus of claim 15, wherein a ratio of the first portion of the quota amount to the second portion of the quota amount is proportional to an historic NPU usage ratio, wherein the historic NPU usage ratio indicates a ratio of an amount of quota for the subscriber traffic flow that was consumed by the first NPU during all communications with the subscriber end station to another amount of quota for the subscriber traffic flow that was consumed by the second NPU during all communications with the subscriber end station.
 18. A method in a network element acting as a gateway in a communications network for intelligent quota management, the method comprising: managing a first quota that is associated with a subscriber traffic flow and that limits an amount of traffic travelling between the network element and a subscriber end station in the subscriber traffic flow, the managing including: assigning a first quota portion to a first network processing unit (NPU), wherein the first quota portion is a portion of the first quota; assigning a second quota portion to a set of one or more other NPUs, wherein the second quota portion is another portion of the first quota; determining to change the distribution of an unconsumed quota amount of the first quota; determining the unconsumed quota amount of the first quota; assigning a third quota portion to the first NPU, wherein the third quota portion is a portion of the unconsumed quota amount of the first quota; and assigning a fourth quota portion to the set of other NPUs, wherein the fourth quota portion is another portion of the unconsumed quota amount of the first quota; and sending a quota consumed report message for the first quota to a remote quota server, wherein the quota consumed report message includes a consumed quota amount, wherein the consumed quota amount indicates an amount of the first quota that has been consumed by the first NPU and the set of other NPUs; wherein the sending the quota consumed report message for the first quota occurs responsive to a redistribution count being equal to a redistribution limit, wherein the redistribution count is a number of times the determination to change the distribution of the first quota has occurred since a last quota provision message for the first quota was received from the remote quota server, wherein a quota provision message includes a quota provision amount, the quota provision amount indicating a quota amount to be added to the first quota, and wherein the redistribution limit is a defined number of times the amount of the first quota that is unconsumed is to be redistributed to a set of one or more NPUs before sending the quota consumed report message to the remote quota server.
 19. The method of claim 18, wherein the third quota portion and the fourth quota portion are assigned in proportion to an NPU usage ratio, wherein the NPU usage ratio indicates an amount of the first quota consumed by the first NPU in relation to another amount of the first quota consumed by the set of other NPUs.
 20. The method of claim 18, wherein the determining to change the distribution of the unconsumed quota amount of the first quota comprises receiving a quota exhausted message from the first NPU, the quota exhausted message indicating that the first NPU has consumed an amount of the first quota portion. 